A Framework for Vision Based Formation Control

Abstract—We describe a framework for controlling and coordinating a group of nonholonomic mobile robots for cooperative tasks ranging from scouting and reconnaissance to distributed manipulation. The framework allows us to build complex systems from simple controllers and estimators. This modular approach is attractive because of the potential for reusability. In addition, we show that our approach to composition also guarantees stability and convergence in a wide range of tasks. There are two key features in our approach. The first is a paradigm for switching between simple decentralized controllers thus allowing changes in the formation. Second, all the controllers use information from a single sensor — an omnidirectional camera. We describe estimators that abstract the sensory information at different levels enabling decentralized as well as cooperative control. Our results consist of numerical simulations as well as experiments on a platform of three nonholonomic robots. Keywords—Hybrid control, formation control, cooperative localization, distributed manipulation, nonholonomic mobile robots.